1. Field of the Invention
This invention relates to an Internet-based, computer-aided system and method that allows the user to quickly and accurately select eyeglass frames, be accurately fitted for eyeglasses, and purchase the proper prescription eyeglasses, which can be accomplished through a kiosk environment with the ophthalmologist or optometrist, and specifically to an Internet-based modeling kiosk that allows for the fitting, selection, sale, and ordering of prescription eyeglasses, that utilizes the Internet to connect the customer, the manufacturer of the lenses, the manufacturer of the eyeglass frames, and the presiding eye doctor, to greatly reduce the overall cost and increase accuracy and speed of purchase, while reducing inventory and space necessary for providing optical services.
2. Description of Related Art
The sale of wearable goods, such as eyeglasses, clothing, and other individually-sized goods through retail outlet stores results in many stores having huge inventories of goods that are required because of the vast differences in the aesthetic tastes and sizes of human beings for wearable goods. The maintenance of such a huge inventory, including the shipping costs, and the sheer volume of storage space required for the goods, results in greatly increased costs in the production and sale of wearable goods.
A system and method for fashion shopping is shown in U.S. Pat. No. 5,930,769, issued Jul. 27, 1999, to Rose, using a programmed computer and other electronic medium, where the customer essentially provides measurements.
U.S. Pat. No. 5,539,677, issued Jul. 23, 1996, to Smith shows a method and apparatus for measuring foot sizes electronically, which includes tracing the outline of the consumer""s foot and an electronic ordering means for ordering shoes.
A method and system for ordering products is shown in U.S. Pat. No. 5,515,268, issued May 7, 1996, to Youda, which allows a customer to select a product from a product list stored in a remote computer system via a communication line.
The manufacture and sale of prescription eyeglasses requires the consumer to typically obtain a prescription from an ophthalmologist, make one or more trips to an optician, where a specific set of eyeglass frames are selected, and the prescription is filled by the optician by grinding special lenses, and then the eyeglasses are subsequently fitted to the user or customer, requiring another trip to the optician.
U.S. Pat. No. 4,991,005, issued Feb. 5, 1991, to Smith, shows a glass frame viewing assembly and method of use, which allows someone to try on various eyeglass frames and observe the result on a television monitor.
U.S. Pat. No. 4,730,260, issued Mar. 8, 1988, to Mori, et al. shows a method for eyewear simulation and a device for its practice. Essentially, a video camera is used that takes a picture of the person and then synthesizes frame information to show different displays of how glasses would look on a person.
U.S. Pat. No. 5,280,570, issued Jan. 8, 1994, to Jordan shows a spectacle image and lens simulating system and method. This system permits a patient to visualize how he or she would look, given a particular choice of spectacle frames.
U.S. Pat. No. 4,852,184, issued Jul. 25, 1989, to Tamura, et al., shows a method and system for spectacles frame image processing and recording, which allows an image of a human face and the glasses frame image to be synthesized by edge processing and color mixing.
U.S. Pat. No. 5,592,248, issued Jan. 7, 1997, to Norton, et al., shows a computerized method for fitting eyeglasses. Digital images of the user""s face and head are enlarged to obtain the most accurate measurements with respect to the bridge and temples.
A system and method enabling shopping from home for fitted eyeglass frames is shown in U.S. Pat. No. 5,983,201, issued Nov. 9, 1999, to Fay. The invention described therein shows a method of providing a customer an opportunity to examine on a PC how the customer would, appear wearing different eyeglass frames, fitted electronically to the customer. The invention incorporates, without the benefit of a technician, the use of a digital camera and scaling device, for providing photographs for scaling distances in the photographs to actual distances to provide customer size. Fay also teaches the use of expensive, stereoscopic photographic equipment for obtaining customer data.
Fay also fails to teach an image fitter with data about frames including 3-D representations of the frames that may be selected. This type of data is critical for a customer to see how the frames may look both on a 3-D representation of the customer""s head from every angle and from one or more photorealistic 2-D representations. Moreover, Fay teaches a system whereby the customer could obtain the glasses directly from the manufacturer. This system is extremely dangerous because severe physical effects could result from wearing poorly fitted prescription eyeglasses.
Also, although Fay teaches the use of color photography, Fay does not teach making a composite of a person""s 3-D image and frame images. Furthermore, Fay does not teach occulting of the glasses by head and hair features, which require high fidelity depth refinement. Also, the use of multiple 2-D images from different views or 3-D imaging is not taught. Moreover, since Fay requires a xe2x80x9cscaling devicexe2x80x9d in obtaining its image, Fay does not teach or suggest a spatially correlated non-gridded image for image composition and presentation.
The present invention, however, is much more accurate and employs the use of a technician when the customer is being fitted at the kiosk. The present invention also includes two or more different 2-D camera shots that may be shot simultaneously for both a 2-D photorealistic view and a 3-D photo textured model. The present invention has a unique method for determining necessary distances and accuracy for frame fitting.
The technician assures that the glasses will be properly fitted, and that proper measurement of critical optometric data is made. In addition, the accuracy of the measurements in the present invention allows a 3-D phototextured image to be more lifelike than that available in the prior art. Also, the 2-D photo composition of the data obtained in the present invention will surpass the realism of the prior art.
The present invention also teaches 3-D modeling of the frames to provide complete information about a pair of frames to the customer. Also, no expensive stereoscopic equipment is needed in the present invention.
An Internet-based system and method for selecting, fitting, and purchasing prescription eyeglasses from a single, small, kiosk, comprising a modeling kiosk for the customer to select and have the glasses fitted while getting a prescription for the proper lenses and for ordering the glasses, said modeling kiosk located at a predetermined location, a plurality of personal computers, one being located at the modeling kiosk, connected to one or more other computers located at the frame manufacturer and the lens manufacturer, all of which are, connected to the modeling kiosk by the Internet.
The modeling kiosk would typically be set up at the ophthalmologist""s or optometrist""s office and would occupy a very small space.
The eyeglass customer""s two dimensional (xe2x80x9c2-Dxe2x80x9d) photos and three dimensional (xe2x80x9c3-Dxe2x80x9d) textured model and logistics data is acquired at the modeling kiosk that typically can be provided in a very small area in the ophthalmologist""s office. The 3-D model facilitates parameter gathering for specific eyeglass specification items required, such as bridge- and temple-distances for the proper fitting of the eyeglass frames. Internet and PC provided data paths from the modeling kiosk node to supply nodes, such as point-of-sale, store: and vendors, and even the customer""s home computer node, are provided via the Internet. Databases, database access, and rendering software are provided at each node to facilitate respective functionality, such as the modeling kiosk data processing, server, privacy/security, customer""s home-computer data viewing, order placement, and order processing.
Provided at the modeling kiosk will be a 2-D photo gathering and 3-D data gathering and reconstruction system, based on the head of the customer, to receive the eyeglass and frames, computer image processing interface hardware, and image recognition software, Internet access and server system for the modeling kiosk, vendor, and customer nodes, software necessary to implement 2-D photorealistic discrete views and 3-D reconstruction from 2-D image data, automatic generation of a database containing 2-D photo images, 3-D models, texture data, Rx (lens prescription) data, and customer data.
The modeling kiosk will also include. Internet browser compatible graphics user (GUI) interface software for 2-D and/or 3-D viewing in order placement.
Specifically, in each modeling kiosk there will be one or more digital video cameras and one or more light projectors projecting one or more high contrast patterns of a known geometry. The placement and attitude relation ships between the cameras and projectors will be known, and all cameras will be in communication with one computer system. In one embodiment, only one projector will be active at a time. Each camera should be capable of high resolution color, or 640xc3x97480 pixels, minimum.
The modeling kiosk and its PC have a unique Internet address. The one or more cameras may be used to gather digital imagery of the subject""s head in two exposures taken with a minimal time interval. A first exposure, taken with general illumination rather than a projected light pattern, is used for 2-D photorealistic imagery and/or to provide texture for a 3-D geometric modeling of a subject""s head. A second exposure is taken with light only from a pattern projector and is used to gather data for a 3-D point cloud for the 3-D modeling of the subject""s head. The 3-D modeling is required for the option of applying photo texture to the image of the subject""s head. If the image requested is a photorealistic 2-D image, then the 3-D head model provides positioning and orientation information and occulting information to construct an image of the frames and the head photoimagery. If more than one pattern projector is used, additional exposures with pattern illumination may be optionally added to the sequence. The kiosk PC will provide for the entry of data and database queries via a server to which uploads of customer data from one or more customers is sent.
The 2-D digital imagery, 3-D customer model and texture patterns are stored on a server. The server may be centralized for convenience and for ease in obtaining data. There are separate options for viewing the images derived from the data obtained. One option provides a set of 2-D photorealistic compositions of digital photos of the subject combined with data of the eyeglass frame of interest. This option may have the computation done by the centralized server and have the 2-D image data transmitted to an optometry browser application on the customer""s PC. This browser application may also provide other optometric data to the customer. As another option, a 3-D phototextured model composite of the customer wearing the frame of interest may be provided. An application on the server may produce a unified model of the 3-D image of the customer and of the frame of interest and transmit it to the customer""s PC. The customer would be able to view the unified model and rotate it. Viewing, moving and rotation of the 3-D model may be accomplished by a function of the browser or with a plug-in, such as a VRML plug-in, to the browser application running on the customer""s PC. On subsequent accesses by the customer, it can be automatically determined whether the customer needs another download of the model or if it is still active in his or her PC.
Databases of vendor items are stored at the centralized server for a specialized market online database. This database is comprised of item specifications, such as frames and lenses, and a 3-D model of each item, including texture patterns. A database query system downloads customer requested items for viewing.
The optometry modeling kiosk system includes creating a database and software program that includes data acquisition, image analysis, the construction of a model, such as the customer""s head and face areas, suitable for measuring eyeglass in frames, a final 3-D model, and then the final database for that customer.
The method for both the 2-D photorealistic model and the 3-D phototextured model contains the steps of data acquisition, image analysis and untextured model construction. Data acquisition is performed by taking photos in general illumination and with projected pattern illumination. The step of image analysis comprises the obtainment of point cloud data for the contours defined by the projected pattern or patterns. Untextured model construction comprises the step of building a polygonal model. One example of this model would have triangular faces with the members of the point cloud as vertices. Thus, this model may be formed by point cloud triangulation. Mapping texture from the general illumination photos onto the textured model completes the process for the 3-D phototextured model.
The above steps may be done once per kiosk visit. Additional steps, described below, are performed when the customer calls up one or more of the images obtained wearing different frames.
For the 3-D phototextured model, the 3-D model of the selected frame is positioned onto the 3-D phototextured mode, and the registered model or models are transmitted to a subject""s PC.
For the 2-D photorealistic discrete view composition, or 2-D/3-D hybrid, the 3-D model of the selected frame is positioned on the untextured model. The superposition of the two models is rendered once for each view for which photos, were taken. For each view, the eyeglass frame pixels not occulted by the head model are transferred to the picture data for that view, replacing the corresponding pixels. The picture data is for all views, modified by the rendering of the frame, are transmitted to the customer""s PC.
As an example, a customer will come to the eyeglass-modeling kiosk, which can be located at an ophthalmologist""s office in a very small space. The customer is placed in front of the one or more cameras and one or more light pattern projectors, approximately at a location ideal for subsequent data analysis, and will be photographed with his head in one or more positions. In each position, for each camera, there will be a photo taken with general illumination and a photo taken with illumination only from one or more pattern projectors. In the preferred embodiment, the head is kept generally motionless in each position, and the general illumination and pattern illumination photos are taken with the minimum reasonably possible time interval between them, so the exposures are taken with as close to identical head positions Was possible. In another embodiment, a rapid sequence of photos is taken, for example, with a video camera in electrical connection with a frame grabber, while the head is rotated with a generally smooth motion.
In this embodiment, illumination would alternate between general illumination and pattern illumination. The 3-D models obtained from the projected pattern photos on either side of a given general illumination photo would be used to obtain a 3-D head model of interpolated position to correspond approximately to the position of the head for that general illumination photo. This method allows a large number of poses tog be captured in a short time with a single camera.
In the fixed head position procedure, the use of one camera and many fixed head positions is equivalent to the use of many cameras and one fixed head position, or to the use of intermediate numbers of cameras and head positions. For example, locating three cameras at positions rotated about the head at intervals of 60 degrees, while using two head positions related by a 30-degree rotation, is equivalent to using one camera and six head positions at 30-degree intervals, or to using six cameras at 30-degree intervals and only one head position. In all cases, one can display six poses at 30-degree intervals in the final 3-D or photorealistic 2-D presentation. This allows the operator of the kiosk system to trade off economy in number of cameras against speed and convenience in data taking. An advantage of using at least two cameras is that quasi-simultaneous 3-D data from multiple cameras for a single head position can be merged directly into a single head model using world coordinates of the 3-D points, since the geometrical relationships between the cameras is fixed and known to high precision. This gives 3-D data over more of the head surface than can be covered by a single view.
The projector(s) must be displaced somewhat from the camera location(s) to provide a baseline for stereoscopic reconstruction of the 3-D surface of the head, yet not so far that the portions of the head covered by the camera and by the projector do not mostly overlap.
If multiple head positions are used, or particularly if data is rapidly taken during continuous head motion, the rapid application and removal of general illumination may be uncomfortable to the subject""s eyes. The need to turn off the illumination may be avoided by using light in mutually exclusive wavelength bands for the general illumination and the projected pattern illumination. The acceptance band of the camera(s) may be switched between these two mutually exclusive bands by, e.g., rotating an optical filter in front of each of the cameras in synchronism with the camera exposures. Such a filter would have two sectors, each one passing only one of the two mutually exclusive wavelength bands. As a practical matter, use of invisible wavelengths, e.g., in the near infrared, for the projected pattern, will interfere the least with natural color rendition in the picture information collected under general illumination. In another embodiment, this wavelength separation may be accomplished by replacing each camera by two similar cameras and a fixed, wavelength-sensitive beamsplitter that locates the two cameras at optically equivalent positions. The use of two cameras allows the two photos, one taken in general illumination, and the other taken with projected pattern illumination, to be taken simultaneously. In the case of photos taken during continuous head rotation, this simultaneity avoids the need to interpolate the position of the 3-D head model between data taken before the general-illumination pictorial data and data taken after it.
The 3-D data around the bridge of the nose, obtained from the pictures taken with projected pattern illumination, will allow the 3-D model of the eyeglass frame selected by the customer to be put into contact with the head in this region. The other points of contact that determine placement of the frame are portions of the external ear that are obscured by the earlobe. To substitute for these points of contact, other points will be determined, through which the temple of the frame must pass. One method of determining such points involves taking additional picture data beyond that described above. In this method, the customer will don a generic eyeglass frame, of a distinctive color, e.g., white, to enable its recognition by automatic image processing. With the customer wearing this frame, additional exposures will be taken, similar to those already taken without the frame. Image processing will find the points at which the temples of the frame disappear, obscured by either earlobes or hair. These points will be transferred to the 3-D models obtained from the exposures taken with the generic frame. By registering these models, on a best-fit basis, with the models obtained without the frame, these reference points will be transferred to the models used to render view of the head. So placed, they may be used to position the models of the frames used to render pictures of the customer wearing his selected frames.
Once the measurement information is obtained, there is a modeling kiosk database generated that includes the customer""s name, address, 2-D photo imagery, head model, and texture database.
The system then generates a database for the optometrist""s use that includes the PD, temple and bridge and fitting information, along with input for the specific glasses, model, textures, and tints, including the prescription.
The customer may select glasses using face software, or may have already selected frames based on the database at the modeling kiosk.
The order is then sent to the frame manufacturer and the lens manufacturer, if they are different, allowing the vendors to fill the orders and send the glasses back to the modeling kiosk for pickup by the customers.
By using the present modeling kiosk system that provides for extremely accurate selection, fitting, and ordering of frames and the accompanying prescription lenses, a significant volume of space is saved, both at the ophthalmologist""s office and at the optician""s office, because of the reduced inventory, while oftentimes reducing the number of middlemen who handle the product, again greatly reducing the overall cost to the customer.
Once the initial database has been established for a specific customer, this database can be maintained by the customer in the customer""s home PC, and subsequent orders for prescription eyeglasses can be made directly by the customer from his home, which would allow him to go to the optician""s to pick up the glasses for the final fit.
Although the system has been shown for fitting of eyeglasses, which are particularly critical as to variations in the shape of people""s head, asymmetrical fit problems, bridge sizes, and nose shape differences, the system may be employed for other wearable goods requiring more or less accurate fitting for the user.
It is an object of this invention to provide a modeling kiosk system and method for wearable goods that must be accurately fit to a customer, such as eyeglasses and eyeglass frames.
It is another object of this invention to provide a photorealistic 2-D image for utmost realism.
It is another object of this invention to provide an Internet e-commerce software and server system that can connect a customer, the manufacturer and other vendors, professional people, such as the optician and ophthalmologist, for selecting, fitting precisely in rapid order for the customer, eliminating the necessary floor space, and inventory-reducing costs.
In accordance with these and other objects which will become apparent hereinafter, the instant invention will now be described with particular reference to the accompanying drawings.